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Abstract — The distortion-rate performance of certain 
randomly-designed scalar quantizers is determined. The central 
results are the mean-squared error distortion and output entropy 
for quantizing a uniform random variable with thresholds drawn 
independently from a uniform distribution. The distortion is 
at most 6 times that of an optimal (deterministically-designed) 
quantizer, and for a large number of levels the output entropy 
is reduced by approximately (1 7) /(In 2) bits, where 7 is 
the Euler-Mascheroni constant. This shows that the high-rate 
asymptotic distortion of these quantizers in an entropy- 
constrained context is worse than the optimal quantizer by at 
most a factor of Ge'^'-^-'''' ^ 2.58. 

Index Terms — Euler-Mascheroni constant, harmonic number, 
high-resolution analysis, quantization, Slepian-Wolf coding, sub- 
tractive dither, uniform quantization, Wyner-Ziv coding. 



I. Introduction 

What is the performance of a collection of K subtractively- 
dithered uniform scalar quantizers with the same step size, 
used in parallel? The essence of this question — and a precise 
analysis under high-resolution assumptions — is captured by 
answering another fundamental question: What is the mean- 
squared error (MSE) performance of a 7^-cell quantizer with 
randomly-placed thresholds applied to a uniformly-distributed 
source? For both (equivalent) questions, it is not obvious 
a priori that the performance penalties relative to optimal 
deterministic designs are bounded; here we find concise an- 
swers that demonstrate that these performance penalties are 
small. Specifically, the multiplicative penalty in MSE for 
quantization of a uniform source is at most 6 in the codebook- 
constrained case and about 6e~^'^^~''' « 2.58 in the entropy- 
constrained case at high rate, where 7 is the Euler-Mascheroni 
constant ||T]. The translation of these results is that the 
multiplicative penalty in MSE for high-rate parallel dithered 
quantization is at most 6 when there is no expoitation of 
statistical dependencies between channels and about 6e^^(^ 
when joint entropy coding or Slepian-Wolf coding |2| is 
employed and the number of channels is large. 

Quantization with parallel channels is illustrated in Fig. [T] 
Each of K quantizers is a subtractively-dithered uniform scalar 
quantizer with step size A. Denoting the dither, or offset, 
of quantizer Qk by a^, the thresholds of the quantizer are 
{{j + afc)A}jgz- One may imagine several stylized appli- 
cations in which it is advantageous to allow the a^s to be 
arbitrary or chosen uniformly at random. For example, with 
parallel quantizer channels, one may turn channels on and 
off adaptively based on available power or the desired signal 
fidelity |3|. Alteration of the lossless coding block could 
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Fig. I. Use of K dithered uniform scalar quantizers in parallel. Quantizer 
Qfe has thresholds {(j + afc)A}jgz, with aj; its ojfset. 

then be achieved through a variety of means ||4]-0. The 
same figure could represent a distributed setting, in which K 
sensors measure highly-correlated quantities (all modeled as 
X)\ with a Slepian-Wolf code [2|| or universal Slepian-Wolf 
code [7|, the sensors can quantize and encode their samples 
autonomously. Variations in the a^s could also arise unin- 
tentionally, through process variation in sensor manufacturing 
due to cost reduction or size reduction; mitigation of process 
variations is expected to be of increasing importance fSl. This 
letter addresses the performance loss relative to deterministic 
joint design of the channels or coordinated action by the 
distributed sensors. 

Collectively, the K parallel quantizers specify input X with 
thresholds U^IIq^{(.7 + afc)A}jgz- One would expect the best 
performance from having {ak}^^^^ uniformly spaced in [0, 1] 
through afc — k/K\ this intuition is verified under high- 
resolution assumptions, where the optimal entropy-constrained 
quantizers are uniform f9l. To analyze performance relative to 
this ideal, it suffices to study one interval of length A in the 
domain of the quantizers because the thresholds repeat with 
a period of A. This analysis is completed in Section [III The 
ramifications for the system in Fig.[T]are made explicit in Sec- 
tion |III1 Section |TV] considers uniform quantizers with unequal 
step sizes, and Section |V] provides additional connections to 
related results and concludes the note. 

II. Random Quantizer for a Uniform Source 

Let X be uniformly distributed on [0,1). Suppose that a 
if-level quantizer for X is designed by choosing K — \ 
thresholds independently, each with a uniform distribution 
on [0, 1). Put in ascending order, the random thresholds are 
denoted {ak]k=i^ ™d for notational convenience, let oq = 
and flA' = 1. A regular quantizer with these thresholds has 
lossy encoder a : [0, 1) — > {1, 2, . . . , K} given by 

a{x) = k for a; e [afe_i, flfe). 

The optimal reproduction decoder for MSE distortion is (3 : 

{1, 2, iv:} ^ [0,1) given by 

/3(fc) = i(afc„i+afe). 
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We are interested in the average rate and distortion of this 
random quantizer as a function of K, both with and without 
entropy coding. 

Theorem 1: The MSE distortion, averaging over both the 
source variable X and the quantizer thresholds {ak}^^i, is 
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Proof: Let L{x \ {flfc}^/) denote the length of the quan- 
tizer partition cell that contains x when the random thresholds 
ai-e {ak}k=i )\ i-e., 

L{x\{ak]'^-^) = length(a-i(a(a:))). 

Since X is uniformly distributed and the thresholds are inde- 
pendent of X, the quantization error is conditionally uniformly 
distributed for any values of the thresholds. Thus the condi- 
tional MSE given the thresholds is E [i^ |{afe}£i^] /12, and 
averaging over the thresholds as well gives _D = E L^] /12. 

The possible values of the interval length, {oi — ai-i}^^, 
are called spacings in the order statistics literature llTO] 
Sect. 6.4]. With a uniform parent distribution, the spacings 
are identically distributed. Thus they have the distribution of 
the minimum, ai. 

\K-2 



faM - {K~l){l-ay 



< a < 1. 



The density of L is obtained from the density of a i by noting 
that the probability that X falls in an interval is proportional 
to the length of the interval: 
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completing the proof. An alternative proof is outlined in the 
Appendix. ■ 

The natural comparison for ([l) is against an optimal ii'-level 
quantizer for the uniform source. The optimal quantizer has 
evenly-spaced thresholds, resulting in partition cells of length 
1/K and thus MSE distortion of 1/(12/1^). Asymptotically 
in K, Distortion ^ is worse by a factor of QK"^ /{{K + 
l)(/'i' + 2)), which is at most 6 and approaches 6 as iiT — !> oo. 
In other words, designing a codebook-constrained or fixed- 
rate quantizer by choosing the thresholds at random creates a 
multiplicative distortion penalty of at most 6. 

Now consider the entropy-constrained ox variable- rate case. 
If an entropy code for the indexes is designed without knowing 
the realization of the thresholds, the rate remains logj K bits 
per sample. However, conditioned on knowing the thresholds, 
the quantizer index a{X) is not uniformly distributed, so the 
performance penalty can be reduced. 

Theorem 2: The expected quantizer index conditional en- 
tropy, averaging over the quantizer thresholds {ak}^^^, is 



R = E[i?(a(X)|{a.}£ii)] = 
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Fig. 2. MSE penalty factor as a function of K. For quantization of uniform 
source on [0,1], K is the number of codewords (Section For parallel 
dithered quantization, K is the number of channels (Section ITlll . 



Proof: The desired expected conditional entropy is the 
expectation of the self-information, — logj P (a(X)). Let L 
be defined as in the proof of Theorem [T] to be the length of 
the interval containing X. Since the probability of X falling 
into any subinterval of [0,1) of length c is c, we have 



R 



E[-log2i] 



(log2^) • K{K-l)l{l-i) 



K-2 



which equals (|2]l by direct calculation; see also ifTTl . lfT2l §4.6]. 
An alternative proof is outlined in the Appendix. ■ 
To compare again against an optimal if -level quantizer, note 
that evenly-spaced thresholds would yield R = log2 K while 
the rate in dU is also essentially logarithmic in K. The quantity 
dU includes the harmonic number iJ„ = X]fc=i ^/^^ which 
has been studied extensively. For example, 

24(nV ly ^ H^-l-Hn+\) < ^ 

where 7 « 0.577216 is called the Euler-Mascheroni con- 
stant [HI. 

Combining ([TJ and ^ while exploiting the asymptotic 
approximation i7„ x 7 + ln(7i + ^) yields 

R - (7-l + ln(if +i))/(ln2) 

and a distortion-rate performance of 
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(3) 



where ~ represents a ratio approaching 1 as K increases for 
distortions and difference approaching as if increases for 
rates. The exact performance from (IT}-(|2) is shown in Fig. |2] 
with normalization through division by j22^'^^. 

III. Parallel Dithered Quantizers 

Let us now return to the system depicted in Fig. [T] High- 
resolution analysis of this system for any number of channels 
K follows easily from the results of the previous section. 

For notational convenience, let us assume that the source 
X has a continuous density supported on [0, 1). Fix A ^ 1 
and consider K uniform quantizers with step size A applied 
to X. Quantizer Qq has lossy encoder ao with thresholds at 



3 



integer multiples of A. The remaining K — 1 quantizers are 
offset by a/jA, i.e., the thresholds of Quantizer Qk with lossy 
encoder ak are at {(j + afc)A}j^04, ...,LA-iJ ■ 

We would like to first approximate the distortion in joint 
reconstruction from {ao{X), ai{X), . . . , aK-i{X)). The first 
quantizer index ao{X) isolates X to an interval aQ^{ao{X)) 
of length A. Since X has a continuous density and A ^ 1, 
we may approximate X as conditionally uniformly distributed 
on this interval. Thus we may apply Theorem [T] to obtain 
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where ~ represents a ratio approaching 1 as A — > 0. The 
average of the joint entropy is increased from (|2) by precisely 
H{ao{X)). Since 

lim H{ao{X)) - h{X) - log, A^^ 0, 

A— !-0 

where h{X) is the differential entropy of X 113], 

1 ^ 1 

R - h{X) + log2 + — ^ (5) 

2=2 

where ~ represents a difference approaching as A ^> 0. For 
a large number of channels K, eliminating A gives 



jj (a) exp{2j2i^2^ ^) n2h(X)2~2B. 

2{K+l){K + 2) 
(h) exp(2(7 - 1 + ln{K + i))) 
2{K + l){K + 2) 

exp(2(7-l))(-^+^)% 2hW.-2i.. 
2{K+l){K + 2) 



(6) 
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where (a) is exact as A 0, (b) is the standard approximation 
for harmonic numbers, and (c) is an approximation for large 
K. This distortion exceeds the distortion of optimal entropy- 
constrained quantization by the factor 6e^^(^ '\ 

IV. Quantizers with Unequal Step Sizes 

The methodology introduced here can be extended to cases 
with unequal quantizer step sizes. The details become quickly 
more complicated as the number of distinct step sizes is 
increased, so we consider only two step sizes. We also limit 
attention to source X uniformly distributed on [0, 1). 

Let quantizer uq be a uniform quantizer with step size 
Aq <S 1 and thresholds at integer multiples of Ao (no offset). 
Let Of 1 be a uniform quantizer with step size Ai ^ 1 and 
thresholds offset by ai, where oi is uniformly distributed on 
[0, Ai). Without loss of generality, assume Aq < Ai. (It does 
not matter which quantizer is fixed to have no offset; it only 
simplifies notation.) 

Mimicking the analysis in Section [III the performance of 
this pair of quantizers is characterized by the p.d.f. of the 
length of the partition cell into which X falls. Furthermore, 
because of the random dither ai, the partition cell lengths are 
identically distributed. 



M = 



(7) 



Let M be the length of the partition cell with left edge at 
0. Clearly M is related to ai by 

ai, if ai e [0, Ao]; 
Ao, if ai e (Ao, Ai). 

So M is a mixed random variable with (generalized) p.d.f. 

/A/(m) = i- + - - Ao), m e [0, Ao]. 

With L defined (as before) as the length of the partition cell 
that contains X, 
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Ao(Ai - iAo) Ai - iAo 
for < £ < Aq. The average distortion is given by 

^Ao 



S{£ - Ao), 



^2 L J 



12 Ai-iAo' 



(8) 



This expression reduces to (|4]i (with K = 2) for Ao = Ai = 
A. Also, it approaches Ao/12 as Ai — > oo consistent with 
the second quantizer providing no information. The average 
rate is 

1 Ao 



R 



E[-log2L] = logsAg-i 



(9) 



2 In 2 2Ai - Ao 

This reduces to © (with K ^ 2 and h{X) = 1) for Aq = 
Ai = A. 

One way in which unequal quantization step sizes could 
arise is through the quantization of a frame expansion [14]. 
Suppose the scalar source X is encoded by dithered uniform 
scalar quantization of Y = {Xcos9, X sind) with step size 
A <C 1 for each component of Y. This is equivalent to using 
quantizers with step sizes 

Ao = A/lcose'l and Ai = A/lsin6'] 

directly on X. Fixing 6 G (0, 7r/4) so that Ao < Ai, we can 
express the distortion dHJ as 

A^scc^ei l-|tan6' 

and the rate ^ as 

Re = log2 A^^ + log2 cos 6 - 
The quotient 
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(10) 



can be interpreted as the multiplicative distortion penalty as 
compared to using a single uniform quantizer. This is bounded 
above by 

which is consistent with evaluating ^ at K = 2. Thus, 
joint entropy coding of the quantized components largely 
compensates for the (generally disadvantageous) expansion of 
X into a higher-dimensional space before quantization; the 
penalty is only an e/2 distortion factor or « 0.221 bits. 
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V. Discussion 

This note has derived distortion-rate performance for certain 
randomly-generated quantizers. The thresholds (analogous to 
offsets in a dithered quantizer) are chosen according to a 
uniform distribution. The technique can be readily extended to 
other quantizer threshold distributions; however, the uniform 
distribution is motivated by the asymptotic optimality of 
uniform thresholds in entropy-constrained quantization. 

The analysis in Section |III] puts significant burden on 
the entropy coder to remove the redundancies in the quan- 
tizer outputs (io, ii, ■ • ■ , iK-i)- This is similar in spirit to 
the universal coding scheme of Ziv fl5l . which employs a 
dithered uniform scalar quantizer along with an ideal entropy 
coder to always perform within 0.754 bits per sample of 
the rate-distortion bound. In the case that the quantizers 
are distributed, we are analyzing the common strategy for 
Wyner-Ziv coding |[T6| of quantizing followed by Slepian- 
Wolf coding; we obtain a concrete rate loss upper bound of 
i log2(6e~^^^~'''-') ~ 0.683 bits per sample when the rate is 
high; this is approached when the number of encoders is large. 
With non-subtractive dither, the randomization of thresholds 
is unchanged but the reproduction points are not matched to 
the thresholds. Thus, the rate computation is unchanged but 
distortions are increased. 

Use of analog-to-digital converter channels with differing 
quantization step sizes was studied in ifTTl . Unlike the present 
note, this work exploits correlation of a wide-sense stationary 
input; however, it is limited by a simple quantization noise 
model and estimation by linear, time-invariant (LTI) filtering. 

Exact MSE analysis of quantized overcomplete expansions 
has proven difficult, so many papers have focused on only the 
scaling of distortion with the redundancy of the frame lfT4l . 
lfT8l - ll20l . The example in Section HVl could be extendable to 
more general frame expansions. 

Appendix 

The proofs of Theorems [T] and |2] are indirect in that they 
introduce the random variable L for the length of the partition 
cell containing X. A more direct proof is outlined here. 

Lemma 1: For fixed thresholds {ak}^J^i, 

k=l 

K 

H {a{X)\{ak}^^i) = - (flfc - Ofc-i) log2 (afc ^ Qfc-i) ■ 

fc=i 

Proof: The quantizer maps interval [ak-i,ak) to k so 

P {a{X) = k I {aj}fs^') = flfe - flfc^i. 

The entropy expression is thus immediate. The distortion 
expression follows by expanding the expectation using the law 
of total expectation with conditioning on a{X): 



E[{X-l3iaiX)r\{a,}fS['] 

K 

= [(X - P{a{X)f I a{X) = k, {a,]fs,'] 
k=i ^ « ' 

■V{a{X)^k\{a,}f-,'). 
V , ' 

(ofc-Ofc-l) ^ 

The theorems are proved by averaging over the joint distri- 
bution of the quantizer thresholds {ai}fSi^, which is uniform 
over the simplex < ai < 02 < • • • < uk-i < 1- 
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